Invariant Information Bottleneck for Domain Generalization

Invariant risk minimization (IRM) has recently emerged as a promising alternative for domain generalization. Nevertheless, the loss function is difficult to optimize nonlinear classifiers and original optimization objective could fail when pseudo-invariant features geometric skews exist. Inspired by IRM, in this paper we propose novel formulation generalization, dubbed invariant information bottleneck (IIB). IIB aims at minimizing risks simultaneously mitigating impact of skews. Specifically, first present causal prediction via mutual information. Then adopt variational develop tractable classifiers. To overcome failure modes minimize between inputs corresponding representations. significantly outperforms IRM on synthetic datasets, where occur, showing effectiveness proposed overcoming IRM. Furthermore, experiments DomainBed show that 13 baselines 0.9% average across 7 real datasets.